Problem Statement:

Reduce the Withdrawal/Deposit ratio by at least 10% to increase the free cash flow that the company has after the GST input has been revised from 18% on platform fee to 28% on user deposits, leading to a significant reduction in free cash flow. The current W/D ratio is at 73% i.e., for every Rs 100 deposited, Rs 73 are withdrawn. If we can bring this ratio down to 65%, we can easily extend the current runway to 2 years.

Overview:

After the recent change in GST regulation, GST is being discharged at 28% on user deposits. Since this is a drastic change for customers, MyTeam11 has decided to bear the GST costs internally, without passing them on to the consumers. This has significantly strained free cash flow and led to a higher-than-usual burn rate (an increase of more than 1500% MOM).
In this document, we will explore a few options for allowing users to convert their winnings to deposits without withdrawing money from their wallets, hence retaining the amount in the system, leading to improved cash flow and reducing the Withdrawal/Deposit ratio by at least 10%.

Hypothesis:

If we show the users a way to convert their winnings into deposits at the time of withdrawals (with some bonus),

then we can reduce the W/D ratio by 10%,

due to people playing more frequently and the reduced cost of multiple (in and out) transactions.

Goal/Objective:

In the current withdrawal flow, once a user has completed KYC, they can click Withdraw Money, Enter the amount (Min. INR 200), and click on the "Withdraw" button to take money instantly into their bank account. In this experiment, we will introduce a new flow where users right before entering the amount, can interact with a widget in the app that will show them the option of converting their winnings into deposits without withdrawing, we'll be testing two versions here one with some bonus and one without bonus.
The goal is to improve the Withdrawal/Deposit rate and encourage users to play more. Overall the objective is to keep the free cash flows healthy, reduce the burn, and increase the spin ratio (Gameplay/Deposit).

Success Metrics:

Worst Case: Absolute decrease in Withdrawal Rates (Withdrawal/Deposit) of 3%

Best Case: Absolute decrease in Withdrawal Rates (withdrawal/Deposits) of 10%

It is important to note that seamless Withdrawals and Deposits are core functions of a real money game. Hence by improving the Withdrawal rates, we don't want to impact other core levers of the business like Day-7 retention rate, Number of queries related to amount withdrawal, etc.

​

Experiment Design:

For the users who are putting withdrawal requests on the platform, we want to introduce an extra step to allow them to convert their winnings to deposits directly, without withdrawing money to the bank account.

We analyzed the current withdrawal rates, and here is a snapshot of the data:

​

Average Withdrawal Requests per day = 1400

1-day Redeposit Rate = 11%

3-day Redeposit Rate = 18%

7-day Redeposit Rate = 23%

TDS-Withdrawal Rate = 43%

GST Rate = 28% flat on each deposit

Average TDS Rate = 6% of Withdrawal Amount

PG Deposit Charges = Charges on each Despoit transaction (approx. 2% of Deposited Amount)

PG Withdrawal Charges = INR 2.8

CSAT score before the GST change = 4.6

CSAT score after the GST change = 3.7

Data Definitions:

• Redeposit Rate = Percentage of users depositing money within a unit of time from their last withdrawal and the deposit amount is <= Last Withdrawal Amount
• TDS-Withdrawal Rate = Percentage of withdrawal transactions attracting TDS out of total withdrawal transactions
• GST Rate = All deposits on a real money gaming platform attract a 28% GST flat rate, so if a user is depositing INR 100, they effectively receive 100/1.28 i.e., INR 78.125 in their wallet and INR 21.875 goes towards GST
• AVG TDS Rate = Average percentage of each withdrawal request going towards TDS
• PG Deposit Charges = Average amount paid towards payment gateway charges for each deposit transaction
• PG Withdrawal Charges = Average amount paid towards payment gateway charges for each withdrawal transaction

​

Summary:

We notice from the data that 23% of the users that withdraw money from the platform, deposit the money back on the platform within 7 days. During this cycle, both the user and the platform incur various costs.

User Costs

• TDS at the time of withdrawal
• GST at the time of deposits

Platform Costs

• PG charges at the time of deposits
• PG charges at the time of withdrawal

Overall,

• The existing customer behavior of withdrawing upon each winning and redepositing leads to significant monetary value destruction and strains cash flow.
• The users are unhappy about the TDS deductions on winnings, as many times they incur losses on the game in the future, which they are not allowed to offset against past winnings and can only claim this while filing ITR.
• Users feel that the platform should bear the cost of GST themselves and it should not be passed to them.

​

By allowing the users to convert their winnings into deposits internally on the same platform, the value destruction can be minimized.

​

Segmentation:

Since we know from the data that users with age up to 30 days, contribute to 68% of the withdrawal rates, we will focus on this cohort of users for this experiment.

Users submitting withdrawal requests - Since the proposed flow will only trigger while withdrawing winnings, it is logical to target the users who are taking the step.

• Control Grooup - no change
• Test Group 1 - Convert the winnings with no bonus
• Test Group 2 - Convert the winnings with some bonus

​

Variation Design:

Current Withdrawal Flow:​

Select "Wallet" from the bottom bar -> Click on "Withdraw Money" -> Enter Amount -> Click on "Withdraw Now" -> Check TDS calculations and bank Details and click "Instantly Withdraw XXX" -> Withdrawl confirmed.

​

This flow will be kept as is for the "Control Group".

Variation Design 1:​

In this variation we allow users to convert their winnings into Deposits and save on TDS for the time being.

User Journey

Select "Wallet" from the bottom bar -> Click on "Withdraw Money" -> A small widget to convert the winnings to Deposits is shown and CTA for "I want to Convert" is highlighted. User Clicks on the CTA -> User enters the amount they want to convert and hits "Convert Now" -> Amount is instantly converted to Deposits - a confirmation with TDS savings is shared.

​

This flow will be used for Test Group 1.

Variation Design 2:​

In this variation we allow users to convert their winnings into Deposits with some bonus that can be used for gameplay and save on TDS.

​

User Journey

Select "Wallet" from the bottom bar -> Click on "Withdraw Money" -> A small widget to convert the winnings to Deposits is shown with added benefits and CTA for "I want to Convert" is highlighted. User Clicks on the CTA -> User enters the amount they want to convert and hits "Convert Now" -> Amount is instantly converted to Deposits - a confirmation with TDS savings and bonus given is shared.

​

This flow will be used for Test Group 2.

​

Audience & Sample Size:​

Sample Size Decision:

Since the average number of withdrawal requests completed per day is about 1400 and the number of unique users submitting these requests is about 1000, we take that as the population size and calculate a sample size per variation to achieve statistical significance. This comes out to be about 278, for safety, we pick 300 for each cohort.

Source: https://www.qualtrics.com/blog/calculating-sample-size/

Duration of the experiment:

Given the number of withdrawal requests per day is about 1400, and 68% of them fall in the test buckets i.e. about 950 requests per day, so we have enough requests according to the sample size to conclude the experiment in 1 day. However, the business experiences some seasonality owing to cricketing calendar schedules. Typically, we have observed that in 7 days at least 1-2 major cricketing events are there. We have also observed a significant influx in deposit requests before a major match and an uptick in withdrawal requests after the game is finished, we would like to conduct this experiment for 7 days to ensure all biases are removed from the test.

​

​

Implementation and A/A Test:

Upon checking with the tech team, we realized that the dynamic widget in the withdrawal flow exists and no new development is required.

Before going live with the experiment, we just wanted to ensure that there were no other biases in the system and hence we conducted an A/A Test. In this test, we did not make any design changes, and just continued with the existing withdrawal flow for all three cohorts and ran it for 3 days.

The data here corroborates that the withdrawal rates hovered at the baseline levels when aggregated for 3 days, hence we can conclude that no other biases in the cohort or the testing platform are present.

​

Experiment Results:

Post A/A test completion we went live with the experiment. Below are the experiment results:

​

We ran the experiment for 7 days to rule out any seasonality biases and upon aggregation, it was evident that Test Group 2 performed the best.

Summary:

• Control Group stayed consistent at a 72.66% withdrawal rate, compared to an annual withdrawal rate of 73%.
• Test Group 1 i.e., Option to convert winnings into deposits without bonus, depicted an absolute improvement in the withdrawal rate of 3.49%, however, the Day-7 retention went down as well by 2.46% (absolute).
• Test Group 2 i.e., Option to convert winnings into deposits with bonus, depicted an absolute improvement in the withdrawal rate of 18.23%. The Day-7 retention also improved for this cohort by an absolute 3.76%.

​

Release Decisions:

Upon further investigation for Test Group 2, when adjusted for bonus awarded, which was 4.1% the improvement in Withdrawal Rate was 14.13%. Adding the saved cost of pay-ins and pay-outs for this cohort, the overall improvement in cash flow was +15.67%.

The p-value for this experiment also shows statistical significance and is a 50% increment to the best-case scenario expectation of 10% improvement to the withdrawal rate baseline.​

​

During this experiment, we kept a close eye on customer complaints or unwanted conversions.

• Test Group 1, out of 102 conversions, 5 customers reached out to revert the conversions and requested to credit the amount back to winnings. CSAT scores remained closer to the baseline.
• Test Group 2, out of 948 conversions, 3 customers reached out to revert the conversions and requested to credit the amount back to winnings. CSAT scores remained closer to the baseline.

Test Group 2 depicted an affinity towards conversion and stayed on the platform for longer with no negative actions than usual.

​

The final decision was to scale the experiment to all audiences.

​

Learnings:

• Rewarding users with bonuses at the time of conversion led to a major push to improve Withdrawal Rates.
• Improved withdrawal rates led to improved Gameplay and Spin ratio.
• CSAT scores remained steady and no observed spike in withdrawal-related customer complaints, except for 1 or 2 odd cases where customers accidentally converted and then later wanted a refund.
• Day-7 retention for customers opting to convert increased as well.

​

Next Steps:

Since it has been observed with statistical significance that,

• We will be scaling this to 100% of the users. No additional tech changes are required as the widget already exists and can be invoked from the server side.

• Monitor the customers converting their winnings for negative actions or customer complaints against conversion.

• In the future, we will also experiment with bonus rates and try to observe if further cost optimization is possible.